There is currently a proliferation of organizational networked computing systems. Every type of organization, be it a commercial company, a university, a bank, a government agency or a hospital, heavily relies on one or more networks interconnecting multiple computing nodes. Failures of the networked computing system of an organization or even of only a portion of it might cause significant damage, up to completely shutting down all operations. Additionally, all data of the organization exists somewhere on its networked computing system, including all confidential data comprising its “crown jewels” such as prices, details of customers, purchase orders, employees' salaries, technical formulas, etc. Loss of such data or leaks of such data to outside unauthorized entities might be disastrous for the organization.
As almost all organizational networks are connected to the Internet through at least one computing node, they are subject to attacks by computer hackers or by hostile adversaries. Quite often the newspapers are reporting incidents in which websites crashed, sensitive data was stolen or service to customers was denied, where the failures were the results of hostile penetration into an organization's networked computing system.
As a result, many organizations invest a lot of efforts and costs in preventive means designed to protect their computing networks against potential threats. There are many defensive products offered in the market claiming to provide protection against one or more known modes of attack, and many organizations arm themselves to the teeth with multiple products of this kind.
However, it is difficult to tell how effective such products really are in achieving their stated goals of blocking hostile attacks, and consequently most CISO's (Computer Information Security Officers) will admit (maybe only off the record), that they don't really know how well they can withstand an attack from a given adversary. The only way to really know how strong and secure a system is, is by trying to attack it as a real adversary would. This is known as red-teaming or penetration testing (pen testing, in short), and is a very common approach that is even required by regulation in some developed countries.
Penetration testing requires highly talented people to man the red team. Those people should be familiar with each and every publicly known vulnerability and attacking method and should also have a very good familiarity with networking techniques and multiple operating systems implementations. Such people are hard to find and therefore many organizations give up establishing their own red teams and resort to hiring external expert consultants for carrying out that role (or completely give up penetration testing). But external consultants are expensive and therefore are typically called in only for brief periods separated by long intervals in which no such testing is done. This makes the penetration testing ineffective as vulnerabilities caused by new attacks that appear almost daily are discovered only months after becoming serious threats to the organization.
Additionally, even rich organizations that can afford hiring talented experts as in-house red teams do not achieve good protection. Testing for vulnerabilities of a large network containing many types of computers, operating systems, network routers and other devices is both a very complex and a very tedious process. The process is prone to human errors of missing testing for certain threats or misinterpreting the damages of certain attacks. Also, because a process of full testing against all threats is quite long, the organization might again end with a too long discovery period after a new threat appears.
Because of the above difficulties, several vendors are proposing automated penetration testing systems. Such systems automatically discover and report vulnerabilities of a networked system, potential damages that might be caused to the networked system, and potential trajectories of attack that may be employed by an attacker.
Before using an automated penetration testing system for running a test of a networked system, a user of the penetration testing system (which may be a CISO of the organization, an administrator, a member of a red team, etc.) typically selects a pre-defined scenario that specifies values for the information items of the test and then launches a campaign based on that scenario. For example, the pre-defined scenario may specify that the attacker is a state-sponsored organization having high expertise and unlimited resources. The pre-defined scenario may also specify that the goal of the attacker is to export as many Excel files as possible out of the networked system. It may also specify that the lateral movement strategy of the attacker is “depth-first”, meaning that the attacker will prefer penetrating deeper and deeper into the tested networked system rather than systematically compromising all network nodes that are closest to its initial penetration point. Similarly, there are many other information items that may be specified by the pre-defined scenario which will affect the execution of any penetration testing campaign based on the scenario.
Pre-defined scenarios are typically delivered by the provider of the automated penetration testing system. The provider may provide an initial library of pre-defined scenarios when delivering the testing system, and then may upgrade it with additional pre-defined scenarios from time to time.
Some providers deliver a scenario editor together with the penetration testing system, with the scenario editor enabling the user to create his own pre-defined scenarios and add them to the library. Optionally, the scenario editor may also edit and change previously-existing pre-defined scenarios.